Oscillatory crossover from two dimensional to three dimensional topological insulators 
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We investigate the crossover regime from three dimensional topological insulators Tes and 
Bi2Se3 to two dimensional topological insulators with quantum spin Hall effect when the layer 
thickness is reduced. Using both analytical models and first-principles calculations, we find that the 
crossover occurs in an oscillatory fashion as a function of the layer thickness, alternating between 
topologically trivial and non-trivial two dimensional behavior. 

PACS numbers: 73.20.-r, 73.50.-h, 72.25.Dc 



Introduction- Recent discovery of the two dimensional 
(2D) and three dimensional (3D) topological insulator 
(TI) state has generated great interests in this new state 
of topological quantum matter [l|, 0, 0, IH, 0, 0] • P^^" 
ticular, Bi2Tes and Bi2Ses are predicted to have bulk 
energy gaps as large as O.SeF, and gapless surface states 
consisting of a single Dirac cone [5]. Angle-resolved- 
photo-emission-spectroscopy on both of these materials 
observed the single Dirac cone linearly dispersing from 
the r point [6, 7]. These materials have a layered struc- 
ture consisting of stacked quintuple layers (QL), with 
relatively weak coupling between the QLs. Therefore, 
it should be relatively easy to prepare these materials 
in the form of thin films, either by nano-ribbon growth 
method [1], or by molecular beam epitaxy 0. In the limit 
when the thickness d of the thin film are much smaller 
than the lateral dimensions of the device, it is natural to 
ask whether or not the resulting 2D system is a 2D TI 
similar to the HgTe quantum wells fT,^. In this letter, 
we investigate this question, and find a surprising result 
that the crossover from the 3D to the 2D TI occurs in a 
oscillatory fashion as a function of the layer thickness d. 

Effective model analysis- We begin by recalling the 4- 
band effective model of the 3D TI introduced by Zhang 
et a/[5,] with the following Hamiltonian: 
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with k± = kx ^ iky, eo(k) = C + Dik^ + ^2^1 and 
A^(k) = Mq ^ Bikl ^ B2k\. This Hamiltonian has 
been successfully used to discuss the property of V-VI 
semiconductor, such as Bi2Ses and ^^2^63 [5]. The 
four basis of the above effective Hamiltonian are denoted 
as |Pl+,T),|P2j,r),|PlJ,i),|P2j4) with the super- 



script ± standing for even and odd parity and t (i) for 
spin up (down). An important feature is that the two 
orbitals Pl^ and P2z have the opposite parities, so that 
the off-diagonal term are linear in kz and k±. Another 
important model is the 4-band effective model proposed 
by Bernevig, Hughes and Zhang (BHZ)[1] for 2D quan- 
tum spin Hall (QSH) insulator, given by the effective 
Hamiltonian 
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and M(k) = Mq ^ ^2'^^- 



+ B2kj 
system are taken as 
, I) . According to Ref . 
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[1], the orbitals El and HI also have the opposite pari- 
ties, similar to the 3D TI model. The finite size effect has 
been studied previously by B. Zhou et al within the BHZ 
model. [10] Another similarity between these two models 
is that in order to describe 2D or 3D TI, we need the 
condition MoPi,2 < or M0B2 < 0, so that the system 
is in the inverted regime [1]. In fact if we simply take 
kz = 0, the Al, Bi and Di terms vanish and 3D TI 
model ([1]) will reduce exactly to the BHZ model for the 
2D TI(|2]). These similarities suggest that it is possible to 
give a unified description of both the 2D and 3D TI, and 
it is helpful to investigate the crossover between them 
when the dimension is reduced by quantum confinement. 
Therefore in the following we would like to consider the 
3D TI model ([1]) in a quantum well or a thin film config- 
uration. For simplicity, here we assume the film or well 
thickness to be d and use an infinite barrier to represent 
the vacuum. 

To establish the connection between 2D BHZ model 
and 3D TI model, we start from the special case of Ai = 
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and turn on later. For Ai = the eigenvalue problem 
of the infinite quantum well can be easily solved at T 
point {kx = ky = 0). The eigenstate are simply given by 

\En{Hn)) = ^sin + ^) |A), with |A) = |P1+ T 

(i)) for electron sub-bands and |A) = |P2-,t (j)) for 
hole sub-bands, and the corresponding energy spectrum 
are ^e(n) = C + Mq + (I^i + Pi) (^)' and Eh{n) = C - 

Mo^{Di-Bi) {^) respectively. Here we assume Mq < 
and Pi > so that the system stays in the inverted 
regime [1]. The energy spectrum is shown in Fig [1] (a). 
When the width d is small enough, electron sub-bands 
En have higher energy than the hole sub-bands Hn due to 
the quantum confinement effect. Because the bulk band 
is inverted at F point (Mq < 0), with increasing d the 
energy of the electron sub-bands will decrease towards 
their bulk value Mq < 0, while that of the hole sub- 
bands will increase towards — Mq > 0. Thus there must 
be crossing points between the electron and hole sub- 
bands. 

Now let us focus on the crossing points between the 
nth electron sub-band \En) and the nth hole sub-band 
\Hn), which occurs at the critical thickness dd, (ic2, • • • in 
FiglU (a). Near one of these critical values of the low- 
energy physics can be obtained by projecting the 3D TI 
model P into the basis l^n,! (i)) and |Pn,T (i))- The 
resulting effective Hamiltonian is nothing but BHZ model 
P with C = C + Di{n7:/df, Mq = Mq ^ Pi(n7r/d)2, 
A = B2 = B2 and D2 = D2. Following the ar- 
gument of BHZ[1], we know that there is a topological 
phase transition between QSH state and ordinary insula- 
tor across the critical point where the gap is closed. The 
above result can also be obtained from the parities of 
the sub-bands \En) and \Hn) at F point. Since now the 
wave function has four components, the parity should be 
determined by both the function sin + ^) and the 
basis I A). Define P as the inversion operator and we find 
that P\E„) = (-l)"+i|i;„) and P\H„) = (-1)"|JT„), 
hence \En) and \Hn) always have the opposite parities. 
According to the parity criterion of Fu and Kane [3], the 
topological property of the system can be determined by 
the product of the parities of all the occupied energy lev- 
els at all time reversal invariant momenta, which is de- 
noted as u. Near the critical thickness dcn^ the band 
gap is determined by \En) and \Hn) at F point and 
all the other energy levels are far from the Fermi sur- 
face. Therefore, due to their opposite parities, the cross- 
ing of \En) and \Hn) will change the total parity u and 
correspondingly the topological property of the system. 
The crossing points can be determined by the condition 

Ee{n) = Eh{n)^ which leads to den = nir^ jM^i' ^ topo- 
logical phase transition occurs at den for each n, so that 
the system oscillates between QSH phase and ordinary 

insulator phase with the period of le = tt^^^, if the 
Fermi energy always stays within the gap. 

Next we consider the effect of turning on Ai term. As 




FIG. 1: (Color online) The energy level versus the thick- 
ness of the quantum well is shown for (a) Ai — OeV-A, (b) 
Ai — l.leV-k. Other parameters are taken from Ref.[^. The 
shaded region indicates the regime for QSH states. The blue 
dashed line in (b) shows how the crossing between \Ei[Hi)) 
and 1^2(^2)) is changed to anti-crossing when Ai is non- 
zero. In (c), the density of \Si) (\S2) has the same density) 
is plotted for Ai — l.leV- A. In (d), the critical thickness 
den {n — 1, 2, • • • ) is plotted as a function of Ai. QSH states 
appear in the shaded region. 



shown in Fig. [T] (b), Ai term induces the coupling be- 
tween \En{Hn)) and |Pn±i(^n±i)), which results in an 
avoid crossing between these sub-bands. However, a fi- 
nite Ai does not break parity, so that the crossing at 
den between \En) and \Hn) with opposite parity remains 
robust. Since the topological phase transition is mainly 
determined by the crossing between \En) and |Pn), the 
system still oscillates between QSH insulator and ordi- 
nary insulator when the thickness d is tuned. Due to 
the hybridization induced by Ai term, two special states 
(each doubly degenerate due to spin), denoted as l^^) 
and |6'2~) in FigH] (b), are formed within the bulk gap 
and well-separated in energy from other sub-bands when 
the thickness d is large. \S^) is in fact the superposition 
of |P2n-i) and \H2n) = 2, • • • ) while l^*^) is the su- 
perposition of |P2n-i) and \E2n) = 2, • • • ). In large 
d limit, the gap between IS*^) and |6'2~) goes to zero and 
the two states become nearly degenerate. In Fig(T] (c), 
the density |^(2:)p for \S^) or |6'2~) is plotted as a func- 
tion of the position z for a quantum well with the width 
d = 20nm, which indicates that l^^^) and |6'2~) are lo- 
calized on the two surfaces of the thin film. One should 
recall that the bulk material is a 3D TI with surface states 
on each surface [5], so that in the large d limit \Si) and 
|5'2~) are nothing but the bonding and the anti-bonding 
state of the two surface states at the two opposite sur- 
faces. Since l^*^) and |5'2~) have opposite parities, the 
topological phase transition can also be understood as 
the change of sequence between these two states. On top 
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of the exponential decay, the wave function shown in Fig. 
[1] (c) also oscillates with a period ~ tt-^^^, which co- 
incides with the oscillation period of the system between 
QSH insulator and ordinary insulator. Although Ai term 
can not eliminate the crossing between 1^^) and \Hn)^ it 
can shift the critical thickness den- The dependence for 
the critical thickness on the parameter Ai is shown in 
FigH] (d), with the QSH regime labeled by the shading 
regions. One can see that when Ai is increased, all den 
(n = 1, 2, • • • ) are shifted to the large values, but the os- 
cillation length doesn't change too much. This indicates 
that the QSH regime will always exist, although the gap 
will be reduced when Ai is large. 

Realistic materials- The above discussion based on the 
anaytical model gives us a clear physical picture of the 
crossover between 2D BHZ model and 3D TI model, and 
in the following we would like to consider about the pos- 
sible realization in the realistic TI materials Bi2Sez and 
Bi2Te3. Here we carried out the first-principles calcu- 
lations with the ESTATE package! 11] and Vienna ab- 
initio Simulation Package (VASP)[12]. The generalized 
gradient approximation of FEE- type is used for exhange- 
correlation potential and the spin-orbit coupling is taken 
into account. For 2D thin film, we construct the free- 
standing slab model with crystal parameters taken from 
the experimental data. For Bi2Ses and Bi2Tes, five- 
atom layers, including two Ei layers and three or Te 
layers, are stacked along z direction, forming a QL[5^. 
There exists strong coupling between two atomic layers 
within one QL but much weaker coupling, predominantly 
of the van der Waals type, between two QLs. Therefore, 
it is natural to regard a QL as a unit for the thin film. 
We will first neglect the lattice relaxation effect which 
depends on the detail of the lattice environment and ad- 
dress this problem in the end. As we have discussed 
above, the topological property can be determined by 
the total parity which has been successfully utilized to 
predict the 3D TI, such as BixSbi-xlS\ and Bi2Ses^,\^. 
Here we apply this method to the thin film with four 
time-reversal invariant points in 2D Erillouin zone(EZ), 
namely f (0, 0), Ml(7r, 0), M2(0, tt), F(7r, tt), as shown in 
the inset of Figl3] (a). The parity of the wave function at 
these points is well-defined and can be easily calculated 
to determine the topological property. 

The calculated band gap at F point and the related 
total parity u for different QLs are shown in the table 
of Fig|2] (c). The first non-trivial QSH phase appears at 
3QLs for Bi2Ses and at 2QLs for Bi2Tes. The gap at F 
point of the non-trivial QSH phase for 2QLs Bi2Tes is 
quite large, of about 148meF, however as shown in Figl3] 
(a), the Bi2Tes is an indirect gap material, and the in- 
direct gap is an order smaller, only ~ lOmeV, similiar to 
the gap of 4QLs Bi2Ses. When the thickness of the film 
increases, the time cost for the ab initio calculation also 
increases rapidly, therefore we perform the tight-binding 
(TE) model calculation based on the maximally localized 
Wannier function (MLWF)[13, 14] from the ab initio cal- 
culation, which has also successfully applied to calculate 




thickness (Qunilayer) thickness (Qunilayer) 



(c) 


BigSeg 






Gap at r (eV) 


Parity v 


Gap at r (eV) 


Parity v 


1QL 


0.710 


+ 


0.391 


+ 


2QLs 


0.098 


+ 


0.148 




3QLs 


0.004 




0.036 


+ 


4QLs 


0.012 




0.005 




5QLs 


0.004 









FIG. 2: (Color online) The band gap and the total par- 
ity are plotted as the function of the number of the QLs 
for (a) Bi2Se3 and (b) Bi2Te3. The calculation is based on 
TB model constructed by MLWF from first-principles calcula- 
tion. The results from the fully first-principles calculation are 
shown in table (c) for 1 ^ 5 QLs. Good agreement between 
the TB model calculation and first-principles calculation is 
found. 

the surface states of Bi2Ses type of materials [s*]. The 
band gap and the total parity u obtained from the TE 
calculation are summarized in Fig. [2] (a) for Bi2Ses and 

(b) for Bi2Te3. The results for the 1 - 5 QLs fit with 
ab initio calculation, which confirms the validity of our 
method. From Fig|2] (a) and (b), it is clear that the sys- 
tem oscillates between QSH insulator and ordinary insu- 
lator, which verifies our previous analysis of oscillating 
crossover. 

When the 2D system stays in the QSH phase, there 
are topo logically protected helical edge states at the ID 
edgeflli [IB, [l7]. To show the topological feature more 
explicitly, we calculate the dispersion spectra of the he- 
lical edge states directly. As examples, here we study 
the edge states of the 2QLs Bi2Tes and 3QLs Bi2Ses 
film along AI direction, as shown in the inset of Figl3] 

(c) . For a semi-infinite system, combining the TE model 
from MLWF with the iterative method, we can calculate 
the Green's function for the edge states directly. The 
local density of states is directly related to the imaginary 
part of Green's function, from which we can obtain the 
dispersion of the edge states. The topological nature of 
the edge states can be determined by the method sug- 
gested by Fu and Kane [3]. As shown in Figl3] (b) for 
2QLs Bi2Tes, there exist one edge state Si which stays 
in the valence band at f point and goes to conduction 
band at M point. Such edge states are in fact the heli- 
cal edge states which can not be eliminated by the local 
time-reversal invariant perturbation. For 3QLs Bi2Ses 
in Fig|3] (d), there are three edge states Ai^2,3 connect- 
ing the conduction and valence band, which guarantee 
the system to be non-trivial. There are also other trivial 
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FIG. 3: (Color Online) Left: the energy dispersion of 2D thin 
film is plotted for (a) 2QLs Bi2Te3 and (c) 3QLs Bi2Se3. 
Right: the energy dispersion of the semi- infinite film with the 
edge along Al direction is plotted for (b) 2QLs Bi2Te3 and 
(d) 3QLs Bi2Se3- The inset of (a) shows the 2D BZ and that 
of (c) is the top view of 2D thin film with two in-plane lattice 
vectors Al and A2. The ID edge is indicated by the blue line 
along Al direction. The insets of (b) and (d) are the zoom-in 
of the energy dispersion near the bulk gap. 

edge states (^2,3 and A4) always staying in the conduc- 
tion band or valence band, which will not change the 
topological property of the system. 

In the discussion above, we have only considered the 
bulk parameters and have neglected the influence of the 
surface lattice relaxation. Since for 2D thin film, the lat- 
tice relaxation always plays an important role in the elec- 
tronic band structure, we would like to address this prob- 
lem in the following. The largest non-trivial gap appears 
for 2QLs Bi2Tes and 4QLs Bi2Ses^ which is suitable for 
experiment to observe. If the thin film is relaxed in vac- 
uum environment, the non-trivial QSH phase could be 
changed to ordinary insulator phase, due to the change 
of the distance between two QLs. The reason is that 
the Van der Waals interaction between two QLs is very 
week and quite sensitive to the lattice environment. This 



also indicates that the lattice relaxation should depends 
strongly on the pressure and the substrate. Therefore, 
here we propose two different ways to overcome the nega- 
tive influence from surface lattice relaxation. One simple 
way is to applied uniaxial compressive stress (along the 
growth direction) to the film. Based on first-principles 
calculation, it is found that only 0.1 GPa stress can re- 
cover the non- trivial gap, which can be easily achieved in 
experiment 0]. Another way is to fabricate the sample 
on the proper substrate. For the material with the pos- 
itive Poisson ratio, when the lattice is stretched within 
the film plane, it tends to get thinner in the perpendic- 
ular direction. Therefore, we can use the materials with 
the larger in-plane lattice constant than that of Bi2Ses 
or Bi2Tes as the substrate. Since in the present paper 
we focus on the principle of realizing QSH insulator from 
3D TI, further study about the effect of pressure and 
substrate will be addressed elsewhere. 

Conclusion- We have studied the crossover between 3D 
and 2D TI in Bi2Ses and Bi2Tes systems. Based on 
both effective model analysis and ab initio calculation, 
we found that it is possible to obtain 2D QSH state by 
confining 3D TI in a quantum well or thin film configu- 
ration with the proper thickness. Up to now, 2D QSH 
effect has only been observed in HgTe quantum wells 
and our work may open up a new way to search for new 
materials with 2D QSH effect. Recently, the high qual- 
ity samples of Bi2S'e3[l^ and ^^2^63 [9] thin films have 
already been fabricated in experiment, so that it is likely 
to observe the proposed phenomenon soon. 
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Note added: During the course of this work, we have 
learned about the work of H.Z. Lu et al^^ and J. Linder 
et a/[21], in which the similar finite size effect has been 
studied within the 3D TI model of Zhang et a/[5.] . 
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